Towards More Efficient Nystrom Approximation and CUR Matrix Decomposition

Symmetric positive semi-definite (SPSD) matrix approximation methods have been extensively used to speed up large-scale eigenvalue computation and kernel learning methods. The sketching based method, which we call the prototype model, produces relatively accurate approximations. The prototype model is computationally efficient on skinny matrices where one of the matrix dimensions is relatively small, but it is inefficient on large square matrices. The Nyström method is highly efficient on SPSD matrices, but can only achieve low matrix approximation accuracy. In this paper we propose novel model which we call the faster SPSD matrix approximation model. The faster model is nearly as efficient as the Nyström method and as accurate as the prototype model. We show that the faster model can potentially solve eigenvalue problems and kernel learning problems in linear time with respect to the matrix size to achieve 1 + relative-error, whereas the prototype model and the Nyström method cost at least quadratic time to attain comparable error bound. We also contribute new understandings of the Nystroöm method. The Nyström method is a special instance of our faster SPSD matrix approximation model, and it is approximation to the prototype model. Our technique can be straightforwardly applied to make the CUR matrix decomposition more efficiently computed without much affecting the accuracy. Empirical experiments demonstrate the effectiveness of the proposed methods. 1 ar X iv :1 50 3. 08 39 5v 5 [ cs .L G ] 7 A pr 2 01 6